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1. Introduction 

Let K be a compact connected Lie group and let if be a closed connected 
subgroup. Let / denote the inclusion of K into K, /* : 6 — > t the induced embedding 
of Lie algebras, and /* : 6* — > t* the dual projection. In this paper we present a 
solution to the following problem. 

(i) Let O be a coadjoint orbit of K. Its projection f*(0) is a if-stable subset of 
t* . Which coadjoint orbits of K are contained in f*(0)7 

Select maximal tori T in K and T in K, and Weyl chambers tl in t* and 
t* in t*, where t and t denote the Lie algebras of T, resp. f. The set f*(0) 
is completely determined by the intersection A(O) = f*(0) fltl, and according 
to a result of Heckman jl2]], A(O) is a convex polytope. It can be described as 
the support of the asymptotic multiplicity function, for which Heckman found an 
explicit formula. However, because the formula involves an alternating sum in 
which many cancellations take place, it is in practice impossible to determine the 
polytope from it. Our goal is to write down as explicitly as possible the inequalities 
defining the polytope. 
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Our results are inspired by a remarkable paper of Klyachko in which he 
solved the case of the diagonal embedding of U(n) into U(n) x U(n). See also 
Fulton's recent survey paper ||. As in [Ell, the proof of our inequalities relies on 
the Hilbert-Mumford criterion for semistability of flags with respect to equivariant 
ample line bundles. A related result describing the "general" faces of the polytope 
was arrived at independently by Brion 

Recall that the restriction of /* to O is nothing but the moment map for the 
action of K on the homogeneous symplectic A-manifold O and that A(0) is its 
moment polytope. (See Kirwan Observe also that O is a symplectic quotient 

of the cotangent bundle T*K. This implies that A((D) is equal to the intersection 
of A(T*K) with an affine subspace, where A{T*K) denotes the moment cone of 
T*K with respect to the K x A-action. We shall in fact write a complete set of 
inequalities for the cone A(T*K). 

We do so in terms of the Schubert bases of the cohomology groups H'(X) and 
H'(X), the canonical homomorphism <f>: H'(X) —> H'(X), and the action on 
H'(X) of a certain subset of the Weyl group of K, which we name the relative 
Weyl set. Here X and X denote the flag varieties of K and K, respectively. Deter- 
mining the relative Weyl set and the matrix of </> relative to the Schubert bases are 
interesting combinatorial problems, which we do not know how to solve in general. 
In many examples, however, they can be solved and we obtain explicit inequalities 
for the moment cone. We treat a few of these examples in detail. 

An intriguing multiplicative version of Klyachko's theorem was obtained by Ag- 
nihotri and Woodward jl| and Belkale 0, who considered conjugacy classes in 
U(n) and found eigenvalue inequalities for products of unitary matrices in terms of 
quantum Schubert calculus. An interesting question is to what extent their results 
can be generalized to the setting of the present paper. 

Hcckman showed that the geometric problem (|) is closely related to the following 
algebraic problem. 

(ii) Let V be an irreducible A-module. As a A-module it breaks up into isotypical 
components. Which irreducible A"-modules occur in VI 

Indeed, assume that O is integral in the sense that it is the orbit through an 
integral point A G t* + and that V\ is the irreducible A-module with highest weight 
A. Likewise, let Vx be the irreducible A-module attached to an integral coadjoint 
AT-orbit O. Then if V^ occurs in V\, O must be contained in f*{0). (The converse 
question — which integral A-orbits inside f*(0) correspond to irreducible compo- 
nents of V\l — is in general much harder, and will be discussed only briefly in this 
paper.) As was pointed out by Guillemin and Sternberg IlCj, Heckman's work 
implies that the following asymptotic version of is equivalent to (fl). 

(ii') For which rational points (A, A) in tl x i* + does there exist a positive integer n 
such that (nX,nX) is integral and the irreducible module V x occurs in V n \l 

Let S denote the set of all pairs of dominant weights (A, A) such that Vx occurs 
in V\, and let S' denote the set of all pairs (A, A) in t+ x t+ which satisfy (Jii 7 ]) - 
Kramer proved in |L7| that S is closed under addition. Therefore S' is equal to 
the convex hull over the rationals of S. The correspondence between problems 
(|) and ( pj 7 ] ) is then as follows: S' is equal to the set of all rational points inside 
A(T*K), and A(T*K) is equal to the closure of S 1 . (See e.g. §| Theorem 7.6].) 
This correspondence plays an important role in the proof of our main result. 
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In the remainder of this paper we will, for convenience, allow / to be a homo- 
morphism with finite kernel. This does not change the nature of problems (|) and 
([?]) in any way, because the sets S and A(T*K) depend only on the Lie algebras 
of K and K. 

Section ^ is a collection of prerequisite resu lts on subgroups and Weyl chambers. 
The main result of this paper, Theorem |3.1.l| , is stated in Section || and proved in 
Section ^. Section ^| contains the examples. In Appendix |X] we review some material 
on flag varieties and Appendix |b] is a compendium of our notational conventions. 

We are grateful to Friedrich Knop for a number of helpful comments. 

2. Subgroups and Weyl chambers 

Throughout this paper K and K denote compact connected Lie groups and 
f:K—>K& Lie group homomorphism with finite kernel. We fix once and for all 
maximal tori T of K and T of K such that T = T n K . This section covers a 
number of auxiliary results which are needed in Section |^ to state our main result. 
Some are elementary, but are included for lack of a reference. In Section 2.1 we 



show that the Weyl group of K can be viewed as a subgroup of the Weyl group of 



if in a number of different ways. In Section 2.2 we discuss how the partition of the 



Cartan subalgebra t = LicT into Weyl chambers induces a partition of the small 



Cartan t = LieT which refines its partition into Weyl chambers. In Section 2.3 we 
work out an example. 

For the purpose of understanding Section ^ the most important parts of this 



section are 2.2.2-2.2.8. On a first reading the reader may want to skim through 



these and through Example 2.3.2, and then move on to Section |3j. 

A comment on our notation: whenever O is any object or structure associated 
with the group K in a natural way, the corresponding object or structure associated 
with K is designated by O. 



2.1. The Weyl groups. The main result of this section is Theorem 2.1.4, which 
says that there is a canonically defined family of injective homomorphisms from 
W into W, which is parametrized by a certain subgroup W of W. Because the 
homomorphism / has finite kernel, K and f(K) have the same Weyl groups. For 
this reason, let us assume in this section that / is injective and identify K with 
f(K). 

If K acts on a set M then the normalizer of a subset M' is the subgroup 
Af K (M') = {k e K | kM' = W }. We say that a subset K' of K normalizes M' 
if K' C J\f K (M'). The centralizer of M' is the subgroup Z K {M') = f) meM , K m , 
which is normal in Af K (M'). (Here K m denotes the stabilizer or isotropy subgroup 
of m.) We will call the quotient 

W K {M')=M K {M')/Z K {M') 

the Weyl group of K relative to M'. Taking M — K, acting on itself by conjugation, 
we have a Weyl group Wk(K') relative to any subset K' of K. In particular, the 
Weyl group of K is W = W K {T). 

2.1.1. Lemma. (i) Let E and F be closed subgroups of K and let E act on K 

by conjugation. If E normalizes F, then it also normalizes the commutator 
subgroup [F,F], the centre C(F), the centralizer Zk(F), and the identity 
component F° . 
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(ii) Let T' be any closed subgroup of T and let L — Zk{T'). Then 

N K (L) = L ■ (M K (L) nM K (T)) = L° ■ (Af K (L) DAf K (T)), (2.1) 
Af K (T') = L ■ (Af K (T') nAf K (T)) = L° ■ (Af K (T') nAf K (T)), (2.2) 
and the inclusion J\[k{T') H A/r-(T) Mk{T') induces an isomorphism 

Nw{T')/Z w (T') W K (T'). (2.3) 
Hence Wk{T') is finite and the order of Wk(T') divides the order ofW. 
Proof. (0) is trivial. 

It is clear that A/if (L) contains L ■ [Nk{L) H A/if (T)). Conversely, consider any 
g 6 A/if (L). Then gTg~ x C L° because L contains T and T is connected. But then 
gTg^ 1 is a maximal torus of L°, so there exists x £ L° such that gTg^ 1 = xTx^ 1 , 
or equivalently, n = x~ 1 g <E Nk{T). But then also n £ L° • Nk{L) C Mk{L). We 
have shown that g — xn with x E L° and n £ Nk(L) ^Nk{T), which proves ( |2.l[ ). 
Observe that Af K {T') C A^(L) by (fl). Hence, by (p7l|), N K {T') is contained in the 
intersection of i° • (A^(i) HA/r- (T)) and N K {T'), which is equal to L° • (N K {T') n 
M K {T)). The inclusion £ ■ (M K {T ) C\M K {T)) C A^(T') is obvious. This proves 
(P^)- The isomorphism (2.3) is Exercice 4 in Bourbaki Ch. 9, §2]. Let us give 



a hint. It is easy to show that 

A^(T') = {N K {T') nM K (T))/T, 
Z W {T') = W L (T) =M l {T)/T = (LnAf K (T))/T. (2.4) 

This implies that N W {T') / Z W {T') = (A^(T') nAf K {T)) /jLnAf K {T)) . The latter 
group clearly injects into Wk(T') and it follows from (f^2) that this injection is 
surjective. □ 



We can apply ( |2.3| ) to the maximal torus T of the subgroup K, Since W is 
a subgroup of Wk(T) in a natural way, the upshot is that W is a subquotient 
of W. But more is true. Let us denote the subgroup Z^iT) by K. This is a 
connected subgroup that contains the maximal torus T. Its root system is equal 
to the subsystem R of all roots a 6 R which vanish on t. Alternatively, K can be 
described as the centralizer of s: K = Zk(s) = Kg, where s is the smallest (closed) 
face of the Weyl chamber which contains tn t+. In other words, s is the face of t+ 
which is perpendicular to R, 

S= f|{e€f+|a(O = 0}. 

Let C = C(K), C = C(K) and C = C(K) be the centres of K, K and K, 
respectively. Observe that C C C and C C C°. The identity component of C is 
the torus whose Lie algebra c is equal to the linear span of s. For the normalizer 
and centralizer of C we have the following result. 

2.1.2. Lemma. 

Z K (C) = Z K (C°) = K, (2.5) 

M K {C) = Af K (C°) = ATk(K), (2.6) 

Af K (f)CAf K (C), (2.7) 

M & {C)=N ic {C°)=M k {f). (2.8) 
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Proof. We have 

Z K (C) C Z k (C°) = Z K (c) C Z K (t) = Z K (f) = K. (2.9) 

On the other hand, C is the centre of K, so K C Zk(C) and therefore all the 
inclusions in ( ]2.9| ) are equalities. This proves ( |2.5| ). By Lemma |2.1.l| (|) we have 
N K {K) C N k {C), because C is the centre of K._ Likewise ,_Af K (C°) C Af K (K), 
because if is the centralizer of C°, and also Mk{C) C J\f K (C°). In sum, we have 
shown 

A/k(<?) c A/V (<5°) c At(#) c A/W (C), 



which proves (2J3). The inclusion (2/7) follows from ( |2 .6|) p lus the fact that Nk{T) 
is contained in Mk{Zk{T)) — Nk{K). The inclusion ((2.§|) follows from ( f2~6| ) plus 
the fact that Af K (K) f) K = Af k (f), □ 

This implies the following relationships among the various Weyl groups. Here 
W = W R {T) denotes the Weyl group of K. 



2.1.3. Lemma. (i) The inclusions J\f R (T) C At (T) and (2.7) induce injective 
homomorphisms 

W< — >W K (T)< — >W K (C a ) = W K (C). (2.10) 

(ii) Let S be a base (set of simple roots) of the root system R. Then 

Afw(R) = ATw(S) K W, (2.11) 

and the projection Mw{R) — * Nw(S) induces an isomorphism 

j B :W K {C)^Nw{S). (2.12) 

Proof. The kernel of the map M K {f) -> W K (f) is Z K {f)nK = Z R {f) = T, so the 
induced homomorphism W — > H^ff) is injective. By ( [2.5| ), the kernel of the map 
A/V(r) -> Bt(C) is so the induced homomorphism W K {f) -> W K {C) is 

injective. This proves (I). 

Observe that is normal in N W {R) and that A/"w (S)C\ W = {1}. If w £ J\f w (R), 
then w5 is a base of ^, so ww5 = 5 for some w £ which implies w £ A/W(5)-ty. 



This proves ( 2.11 ). Inverting the isomorphism (2.5) (with T' = C) and using (2.4) 



yields an isomorphism 

W K {C) <—» Nw{C)/Z w (C) = N W {C)/W. (2.13) 

We assert that Afw{C) — Nw(R). This follows from the observation that Afw(C) = 
N\y{C°) is the set of Weyl group elements which preserve c, the intersection of 



the root hyperplanes defined by R. Hence we obtain from (2.13) an isomorphism 



W K {C) = N W {R)/W. By (|2.1l|) there is a canonical projection Afw(R) -> Afw(S), 



which induces an isomorphism J\fw(R)/W = AV(5). This proves (2.12). □ 



The base 5 is unique up to an element of W, and a different choice of base 



conjugates the isomorphism (2.12): jws( w ) = wj§(w)w 1 for w £ W and w £ 



Wk(C). We can summarize these results as follows. 

2.1.4. Theorem. A choice of a base S of the root subsystem R gives rise to a 
diagram of canonically defined homomorphisms 

W <—> W K (f) <—> W K (C) M W (S) <— > N W {C) <^-» W. (2.14) 



(i 
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Hence the action of W on T extends to an action on the torus T which preserves 
the intermediate subgroup C and the base S . A different choice of base changes 
the lifting homomorphism jg by conjugation with an element of W — Zw{C). 
Therefore the extension of the W -action from T to C is unique] the extension from 
C to T is unique up to an element ofW. □ 

Via the isomorphism jg we can regard W as a subgroup of W. The image 
j§{W) depends on S, but the subgroup j§(W)W = Wj§(W) does not. We usually 
abbreviate j§ to j when the base S is understood. 

2.1.5. Corollary. The inclusions t — ► C — ► t and the dual projections t* — » c* — ► t* 
are W -equivariant (with respect to any choice of S) and W -invariant. The set of 
projected roots f*(R) C t* is W -stable. The embedding K/T — > K/T induced by f 
is W -equivariant. □ 

2.1.6. Example. Observe that W — {1} if and only if K contains a regular ele- 
ment of K. This is for instance the case if K is the unit component of the fixed-point 
group of an automorphism of K. (See Bourbaki ||, Ch. 9, §5.3].) If W = {1}, then 
S = t+, K = T, and C = C° = T, so ( pH| ) amounts to 

W C W K (T ) C W K {T) = A/W(0) = W, 
where the inclusions are canonical. 

2.2. The relative Weyl set. The Weyl chambers of the Cartan subalgebra t form 
a simplicial subdivision in the sense that they cover the whole of t, every chamber 
is a simplicial cone, and the intersection of any two chambers is a face of each. In 
this section we study how the subdivision of t into chambers induces a subdivision 
of t which is W-invariant and refines the chamber subdivision of t. 

By a face of t we mean a closed face of any Weyl chamber in t. If s is a 
face, let R B be the set of all a £ R such that a > on s. Let g and f) be the 
complexifications of t, resp. t. Then the subalgebra p s — f)ffi© aeflr Qa is parabolic, 
and the correspondence s i— » p 5 is a bijection between faces of t and parabolic 
subalgebras of g containing f). We denote the relative interior of a polyhedral 
subset Q of a real vector space by Q° . 

2.2.1. Lemma. (i) R C f*(R). 

Let 5 and s be faces of t and t, respectively. 

(ii) s Pi t C s if and only if (s H t)° fl S is nonempty. 

(iii) If S fl t C I then ws (1 1 — s fl t for all w G j(Ws)W, where W is the Weyl 
group of K — 2^-(/(T)), and j is any lifting homomorphism as in Theorem 
2~H . 

Proof. (0) follows immediately from the fact that the root-space decomposition 
9 = f) © ©qG-R q ^ s a refinement of the weight-space decomposition of g relative to 
the subtorus T: 

3 = 0® Ox, 

AeA*\{o} 

where A* denotes the weight lattice in t* and — 0y»( Q ) = ^ 0a- 

Let 5 be any face of t. The subset {a|a>Oonsnt}ofi? contains R 5 and is 
therefore of the form i? r , where r is a face of s. It is clear that r is the smallest face 
of t which contains Sdt, and therefore (s D t)° C r°. 
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Now assume that (s nt)°fls is nonempty. Pick A G (s n t)° (~1 s. Then A 6 r°, 
so a root a is in R t if and only if a(X) > 0. Furthermore A G s, so if a G i?g 
then a (A) > 0. According to (|) we can write a = f*(a) with a E R. Hence 
a (A) = (5(A) > 0, which implies a G R x - Therefore, if \± is an arbitrary element of 
5 fl t, then a(/2) = ct(fj) > 0. This shows that s n t is contained in s. Conversely, if 
s fl t G s it is obvious that (s n t) n s is nonempty. This proves (|i|) . 

( |iii| ) follows from the fact that acts trivially on t and that /* is W-equi variant. 

□ 

Select a positive Weyl chamber t+ in t. Among all chambers of t, those which 
intersect t + in a cone of maximal dimension are of special importance. 

2.2.2. Definition. A Weyl chamber t+ in t is compatible with t+ if dim t+ (~l t+ = 
dim t+ . 

Equivalently, t+ is compatible with t+ if there exists a £0 m the interior of t+ 
such that a(£) > for all a G S and for all £ G t sufficiently close to £o- It is 
obvious that compatible chambers exist: pick an arbitrary set of positive roots R+ 
in R, pick £ 6 t+> an d let R' + be the set of a G R such that a(£o) > 0. Then 
R + = R + U R' + is a set of positive roots, and the chamber in t which is positive 
with respect to i?+ is a compatible chamber. Moreover, every compatible chamber 
arises in this way. 

Henceforth we shall fix chambers t + and t + such that t + is compatible with t + . 
We denote the corresponding positive Borel subgroups of G = K and G = K c by 
B and B, respectively. 

2.2.3. Remark. This choice entails a choice of bases of the root systems R and R, 
and hence also of a base of R and a lifting homomorphism j as in Theorem [2.1.4 



For arbitrary w G W we define i w to be the cone wt+ f) t. The following result 
gives a set of spanning vectors for the cones dual to t w (and hence inequalities for 
the Un). Let C denote the cone spanned by the positive roots (See Appendix 
[b] for our conventions regarding roots and weights.) 

2.2.4. Lemma. For all w GW the cone t w is dual to the cone f*(wC). 

Proof. Let £ e t Then f*(wa)(i) > for all a G R+ if and only if a(w- 1 |) > 
for all a G R+. This is equivalent to w^ 1 ^ G t+, i.e. £ G wt + D t = t M . □ 

2.2.5. Definition. The compatible Weyl set W com is the set of all w G W such 
that wt+ is compatible with t+. 

Clearly 1 G W com because t+ is assumed to be compatible with t + , and WWcoin = 



Wcom because of Lemma 2.2.l| (|iij). Here are some further helpful properties. 



2.2.6. Proposition. Let w G W com . Then 

(i) tww = tw and f*(wwC) = f*(wC) for w G W, 

(ii) tw C t + and C C f*(wC), 

(hi) {f*)- 1 {R+) n R C W R+ and (/*)- x (7?_) n R C 

(iv) 7?+ C f*{wR+) and R_ n f*{wR+) = 0, 

(v) B = wBw" 1 n G, 

( y i) f*( w C) * s a proper cone: f*(wC) fl —f*(wC) = {0}. 
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Proof. (|) follows from Lemma [2.2.1 ( ip) . 

The first statement in is proved by setting s = wt + and s = t+ in Lemma 
2.2.l| (|ii|). The second statement then follows from Lemma 2.2.4. 



Consider a 6 R + and a e R such that f*(a) = a. Pick £ £ wt + flt+; then £ can 
be written as with £ £ t + . Hence 

(w-'am = «K) = /* (a)(|) - 5(1) > 0, (2.15) 

and therefore w a e that is a 6 wR+. This proves that n 
R C wR+. If a is negative, we get (w~ 1 a)(^) < in (2.15), so we see that 
(FY^iR-) n R C This proves ®. 

(|iv| ) follows immediately from ([h|) and Lemma |2.2.1 ( i|) . 

The first assertion in (Q) implies that b C u>b and therefore B C wBw^ 1 fl G. 
This also shows that wBw~ l (~l G is a parabolic, and hence connected, subgroup 
of G, so to prove the reverse inclusion we need only show that uib (~l jj C b. This 
amounts to showing that the parabolic subalgebra wb PI g contains no negative root 
spaces, which follows from the second statement in (Q). This proves (|vj). 

Because of Lemma 2.2. 4| the cone f*(wC) is proper if and only if dimtu, = dim t. 
The latter is true because t w = wt+ P\ t+ by ([]) and dim wi+ n t+ = dim t by 
assumption. This proves (|vi|). □ 

Statement |vi[ ) obviously fails for any w € W such that t w is not top-dimensional. 
On the other hand, (|i|)-(0) are true, with exactly the same proof, under the more 
general hypothesis that (~1 1+ be nonempty. Therefore @-(Q) are necessary, but 
not sufficient, conditions for u; to be a compatible element. The proof also shows 
that @ fail when t w is not contained in t+. (When t^, is contained in the 
boundary of t+ then (E3) is true but (pli[)~(|v|) may fail, depending on w.) 

According to (|) the top-dimensional cones t w are labelled by the left quotient 
W\Wcom- K is well-known that every coset Ww contains a unique element of 
minimal length, called its shortest representative. A more convenient set of labels 
is obtained by selecting from each coset the shortest representative. 

2.2.7. Definition. The relative Weyl set W Te \ is the set of shortest representatives 
of the left quotient T / K\W / com . For v S W le \ the cone ty is called a cubicle in t + . 

The sets W com and W rc \ depend of course on the choice of t+ and t+. We 
emphasize that in general neither W com nor W re i are subgroups of W. We now 
show that the collection of cubicles forms a well-behaved conical subdivision of t+ 
which is in one-to-one correspondence with W Te \. 

2.2.8. Proposition. (i) The chamber t + is the union of the cubicles, 

t+= (J tv, (2.16) 

nEW rc , 

and the intersection of any two cubicles is a face of each. 
(ii) If t° t (~1 t V2 ^0 for some v± and V2 6 W rc \, then Vi = V2- 
(hi) W Tei = {1} if and only if C = f*(C). 



Proof. The inclusion Uuew ^ — ^+ f° uows from Proposition 2.2.6| (|ii|). Because t 
is the union of its chambers wt+, the positive chamber t+ of t is the union of all 
cones tw with t^ n t+ ^ 0. From this finite union we can clearly delete those cones 
whose dimension is less than dimt, which leaves only the cubicles. The fact that 
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the intersection of two cubicles is a face of each follows from the corresponding fact 
for the subdivision (J 



If I?, n t V2 ^ 0, then 



The cone i Vl — i V2 has an open relative 



of t. This proves 
ti> 2 by 

interior in t + and furthermore W| = Wi — W for an open dense set of £ £ t° + . This 
implies that there exists £ in v\t+ P\ t+ = V2t+ fl i+ such that We = W . This in 
turn implies that the two faces V\t+ fl c and «2<-+ H c of t are equal to one another. 
Let w = v^v^ \ then w maps the chamber i>it + to the chamber «2t+ and therefore 
must fix their intersection vit + Dc — V2t+ n c. But then w fixes c, so w £ W, i.e. 
i>2 £ ViW . As «i and «2 are both shortest representatives of their cosets modW^, 
we conclude that v\ — V2 ■ This proves . 

Dualizing the condition C = f*(C) we obtain t+ = to,. The latter condition is 
equivalent to the subdivision ( 2.1 6| ) consisting of one cubicle only. According to 
this means that Wrci = {!}■ This proves . □ 



We show in Section 3T that the subdivision ( 2.16 ) is symmetric under the stan- 
dard (duality) involution of t+ . 

Observe that the cubicles are proper cones if and only if TnC is discrete. Indeed, 
every cubicle lv contains the linear subspace tfl c = Lie(TnC). On the other hand, 
c is the largest subspace contained in ut + for all v, so if tfl c = 0, then is a proper 
cone. 



It is clear from Proposition 2.2.8 that by intersecting all the chambers in t with 
chambers in t one obtains a subdivision of the entire Cartan subalgebra t into poly- 

1J - f v u>t+ and which, by Corollary 



hedral cones which refines the subdivision t 
2.1.5| , is VT-invariant. 



An illustration of Propositions 2.2.t and 2.2.8 can be found in Section 2.3. For 
the applications in Section ^ it would be highly desirable to find the inequalities that 
describe each of the cones f*(vC), or equivalently, find an explicit set of spanning 
vectors for each of the cubicles t^. This we have not been able to do in general. 

We conclude this section with some miscellaneous observations. Let W ' be the 
set of all w e W such that (/*) _1 (R+) n R C wR + . By Proposition |2. 2. 6| and the 
remarks following it, W' includes W com as well as all other w such that l w contains 



interior points of t + . Put R' = f] w&W i wR + and C = f] 



\w£W 



wC. 



2.2.9. Proposition. (i) R + C f*(R'). 

(h) c = r(C). 

(iii) S^vSDRfor all v e W Tel . 

Proof. (|) follows immediately from the definition of R' . 

For any subset 6 of a vector space £ we define 6 V to be the cone in <£* consisting 
of all functionals which are nonnegative on 6. Then &\ n 62 = (61 U &2Y for all 
subsets 61 and 62- Applying this observation to ( 2.16 ) we obtain C = {~\ veWrel t^- 
By Lemma 2.2.4 this implies C = C\ veW f*(vC), and hence f*(C IC \) C C. The 
reverse inclusion follows from (|) . This proves (|]) . 

For w £ W let c w be the cone wt+ (~l c. We assert that for all w £ W 

dimc„, = dime wS (~l R is a base of R. (2-17) 

Indeed, if dimc^, = dime, then c is the linear span of c^,. Furthermore, PI R 

is clearly a set of positive roots for R. Let 7 £ wR+ (~l R and write 7 = wj3 with 
(3 £ R + . Let a £ S be a simple root that contributes to (3. Then a > on t+, so 
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wa > on c w . On the other hand, 7 = wj3 E R, so w/3 vanishes on c. This is only 
possible if wa vanishes on c w . Hence wa vanishes on c and therefore wa S R. We 
conclude that 7 is a linear combination with nonnegative coefficients of elements in 
wS n R, so this set forms a base of R. Conversely, if wS n R is a base of R, then c, 
resp. c w , is equal to the product c x t>, where c is the centre of 6 and is the linear, 
resp. positive, span of all fundamental coweights that are perpendicular to wS (~l R. 



Hence dime™ = dime, which finishes the proof of (2.17). 

Now observe that dimc„ = dime because v is compatible. We infer from (2.17) 
that vSDR is a base of R. Therefore vSCiR = for some v G W, or equivalently, 
wvS fl R = S. This implies that 

wvR+ H R = R+. (2.18) 



Take any a G If va G i?, then uwa G R+ because of (2.18). If va is not in 
R, then va G i?+ if and only if wva G R+, because w G W permutes the elements 
of R + \ R + . This shows that the length of v, which is equal to the cardinality of 
vR+ D R-, is greater than or equal to the length of wv. But then w = 1, because 
v is the shortest element in Wv. The result is that vS C\ R = S. □ 



The condition 
proof shows that ( 



lii) is not sufficient for v to be in W rc \ for the following reason. The 
iii ) holds for any v G W which is the shortest W-representative of 
a w G W such that dime™ = dime. Let us take w such that t w is compatible with 
wi+, where w is an arbitrary element of W. Then dimc lu = dime, but if w ^ 1 
then w $ Wj-ci, and therefore the shortest representative of w is not in W Te \. 

2.3. Plethysms. Interesting examples are provided by representation theory. A 
unitary representation of an arbitrary compact connected group K on a finite- 
dimensional Hermitian vector space V is nothing but a homomorphism / : K — > A", 
where K is the unitary group U(V). Note that every nontrivial representation has 
finite kernel if K is simple. We begin by stating some generalities concerning this 
class of examples and then work out a simple case in more detail. 

Let V = be the weight-space decomposition of V and let n(//) denote 

the multiplicity of a weight /i G A*. Then ^2n(p) = n = diml^. The centralizer 
K = Zk{T) and its centre C are given by 

where C M denotes the one-dimensional centre of U(y /i ). Let us choose a maximal 
torus T of U(y) which contains f(T). This boils down to a choice of (unordered) 
bases £> M = {e^, , ■ ■ ■ , e nU)} °^ tne we ight spaces V 1 . Let S be the basis 

B = ]jB" (2.19) 

of V. Then the Lie algebra I of T can be identified with the real span of B by 
sending h G t to the vector 

E M^K- (2.20) 
The Weyl group W = S n is the group of permutations of B; Mw(C) consists of those 



permutations that preserve the decomposition (2.19); and W = Zy/{C) — S n ^ 



consists of those permutations that map each B^ into itself. Let us identify t with t* 
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by means of the standard inner product on t determined by the basis B. Then the 
root system R = R v is the set of all vectors of the form — ef, where (/x, k) ^ (y, I), 
and the subsystem R consists of the vectors — ef. 

Observe that R ^ and W ^ {1} as soon as V has a weight of multiplicity 
greater than 1. For any such representation V all points in f(K) are singular 
considered as points in U(F). 

The matrix of /* (relative to the coroots in t and the basis B in t) and the set 
of projected roots f*(R) C t* can be read off easily from the weight diagram of V. 

2.3.1. Lemma. Put e^ = 53I=i e k f or a ^ we ^ghts /x of V . Then the inclusion 
: t ^ c ^ t is given by 

/,(fc) = 5>(fc)e". 

Hence f*{e£) — fJ- and f*{eft — e\) = [i — v for all (/x, k) and (v, I). 

Proof. For all (/x, k) we have f*(h)(e£) = n(h)e^ because has weight /x. There- 
fore, according to the identification ( 2.20Q , f*(h) is equal to X^M(^) eM - 



Dually, we have 

<c^\ h\ — I c v f (h\\ - , 
BDf*(e£) = n. □ 



</* (eg), h) = {el f.(h)) = V v{h){el e») = »(h), 



According to the discussion following Definition 2.2.2, a compatible pair of cham- 
bers tf and t+ is specified by a set of positive roots in R, i.e. an ordering of each 
of the bases z5 M , and by a set of positive roots in R, which implies an ordering =^ 
on the weight lattice A*. This gives rise to an ordering on the set of pairs (fJ-,k) 
defined by (/i, k) > (is, I) if \x ^= v and (ji, k) > (/x, I) if k > /. In terms of this 
ordering R + is given by the set of all eg — e\ with (/x, k) > (l>, I). 

2.3.2. Example. Take K — SU(3) and consider the unitary irreducible represen- 
tation V\ with highest weight A = 2tti + tt-2 , where ffi and 7T2 are the fundamental 
weights of K. Then dimV^ = 15, so this representation defines an embedding 
/: K — » K, where K = U(15). We number the weights as in Figure [ij This order- 
ing specifies a positive chamber t+ in K which is compatible with the (standard) 
positive chamber tf in A". We denote the corresponding bases of the root systems 
R and R by S and S, respectively. 



Th e projected roots are easy to compute from Lemma 2.3.1 , and by Lemma 
2.2.4 the cubicles in t+ are then determined by the hyperplanes perpendicular to 



the f*{a). The matrix of /* relative to the bases of fundamental weights in SU(15) 
and SU(3) is 

'2 556757655523 1 N 
1022253454352 2, 



(2.21) 



The projected roots and the cubicles are shown in Figure where we have used 
the trace form to identify t with its dual. The arrows, with multiplicities, indicate 
the images of the 105 positive roots. If an arrow occurs as the image of one of 
the 14 simple roots, the number of times it so occurs is written in parentheses. 
Three positive roots are mapped to 0, namely 014, as and aio, so that S = R+ = 
{cti, as, aio} and dim (7 = 14 — 3 = 11. 




Figure 2. Peacock's tail with 105 feathers. Projected positive 
roots f*(R+) and cubicles t v for embedding SU(3) — > U(15) de- 
fined by dominant weight A = 2tti + tt2 



The centralizer K = Zk{T) — Zk{C) is a group of block-diagonal matrices with 
nine 1 x 1-blocks and three 2 x 2-blocks and hence is isomorphic to U(l) 9 x U(2) 3 . 
Its Weyl group is W — (Z/2Z) 3 ; it is generated by the simple reflections S4, s$ and 
sio, and acts by permuting rows and columns of each of the 2 x 2-blocks. The group 
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Nw(C) is isomorphic to Sg x (Z/2Z) 3 x S3. The copy of Sg acts on K and its centre 
C by permuting the 1 x 1-blocks, the copy of S3 permutes the 2 x 2-blocks, and 
the copy of (Z/2Z) 3 permutes the rows and columns of the individual 2 x 2-blocks. 
The group Afw(S) is isomorphic to Sg x S3. It consists of all permutations w in 
W = S15 that map the sets {1, 2, 3, 6, 7, 12, 13, 14, 15} and {4, 8, 10} into themselves 
and satisfy w(i) = i + 1 for i = 4, 8, or 10. 



The homomorphism j : W — > Nw(S) is found as follows. By Theorem 2.1.4, 
for each w G W the image of w under j is the unique w € Afw{S) such that 
f»(w£) — wf*(£) for all £ S t. In fact, in this condition it suffices to take a single £ 
in the interior of one of the cones i„, e.g. £ = 65^ + 5ct2 G ti. An easy calculation 
now yields that on generators j is given by 

j(h) = (1 6)(2 12)(4 8) (5 9)(7 14)(13 15), 
j(s 2 ) = (1 2)(3 7)(6 13)(8 10)(9 11)(12 15). 

The other seven natural embeddings of W into W are obtained by conjugating j 
with any of the transpositions (4 5), (8 9), (10 11), or a product thereof. 
The relative Weyl set consists of the four elements 

vi = (6 7) (12 13), V2 = l, U3 = (2 3)(4 6 5)(7 8 9)(10 12 11)(13 14), 

v 4 = (2 3 6 5 4) (7 8 9 12 11 10) (13 14), < ~ 2 ' 23 ' 1 

which have length 2, 0, 8 and 10, respectively. The cubicles are labelled accordingly 
in Figure ||; the cubicle ti = t + fl t is shown in dark grey. We determined W rc i by 
picking elements in the interior of each cubicle, namely £1 = 35^ + 2a~2 , £2 = 
Qa\ + 5ct2, £3 = 5(5^ + 6S2 , and £4 = 2a\ + 3&2, and reflecting them into the 
positive chamber t+. 

This example disabused us of some overoptimistic notions we entertained. For 
instance, the subgroup /(SU(3)) consists of highly singular points in U(15). More- 
over, the small Cartan t has quite high codimension in c. Notice also that the cones 
tv come in two different shapes. Furthermore, a.2 is not in the image of 5, so S is 
not a subset of f*(S), although R + is a subset of f*(R + ) by Lemma p. 2.1 . Lastly, 



the relative Weyl elements do not preserve the root system R, so W Te \ is not a 
subset of Afw{C). 

3. Main results 

As before, let K and K be compact co nnec ted Lie groups and / : K — ► K a ho- 



momorphism with finite kernel. In Section 3T we state our main result, a complete 
set of inequalities for the moment cone of the cotangent bundle T* K, considered 
as a K x iiT-manifold. We deduce from this complete sets of inequalities for the 
moment cone of T*(K\K) (considered as a K- manifold) and for the moment poly- 
tope of every coadjoint orbit of K (considered as a K- manifold) . As explained in 
the introduction, this is equivalent to an asymptotic result on the decomposition 
of an irreducible if- modu le into irreducible if-modules. This and other corollaries 
are stated in Section p]2 . The inequalities we obtain are in general highly overde- 



termined. In Section 3.3 we explain how to prune them to a more manageable set 



of inequalities. We defer the proofs to Section |J. Finally, in Section 3^ we discuss 
the "self-duality" of the moment cone of T*K and explain it in terms of the action 
of the longest elements of W and W . 
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3.1. Schubert cycles and the moment cone. Recall that T and T denote max- 
imal tori of K, resp. K, such that f = TDK. Let R C t* and R C t* be the respec- 
tive root systems and W and W the associated Weyl groups. Choose bases (sets of 
simple roots) S in R and S in R such that the corresponding (closed) posit ive Weyl 
chambers t+ C t and t+ C t are compatible in the sense of Definition 2.2.2] . Denote 
the corresponding dual Weyl chambers in t* and t* by tl, resp. tl. Consider the 
symplectic manifold T*K on which K acts by left multiplication and K by right 
multiplication. Let us identify T*K with K x fi* by means of left translations; then 
the action of if X if is given by 

With respect to the standard symplectic structure on T*K this action has a moment 
map $ = x $ K : T*K — > I* x J* given by 

= /*(*£)> = -e- (3.1) 

According to |^2|, Theorem 7.6] the set 

A(T*K) = <&{T*K) n (i% x t+) 

is a rational convex polyhedral cone, called the moment cone of T*K. Our main 
result is a set of inequalities describing this cone. 

3.1.1. Theorem. Let (A, A) e t* x t+. T/ien (A, A) G A(T*K) if and only if 

w^Xe Ti-w^X + vC) (3.2) 

for all triples (w,w,v) £ W x W x Wrel smc/i i/iai 

^(w^)^^) 7^ 0. (3.3) 

The proof is in Section ^. First we explain the statement and point out a number 
of consequences. Recall that C C t* denotes the root cone, i.e. the cone spanned 
by the positive roots R + . The relative Weyl set W re \ is the subset (not subgroup) 



of the Weyl group W defined in 2.2.7. For each w £ W and v £ W Te \ the set 
f*(—w~ 1 X + vC) is a polyhedral cone in t* with apex /*(— w~ l X), and so ( |3.2[ ) 
represents a finite number of linear inequalities. For instance, if if is contained in 



tf , then by Proposition [2.2.q , W rc i = {1} and f*(C) — C, the root cone of K. 

The condition (3^3) is of a homological nature. Let X = K/T be the flag variety 
of K. The Weyl group W acts on the homogeneous space X by right multiplication, 
w(kT) = kw~ l T . (We denote a Weyl group element and any of its representatives 
in Nk(T) by the same letter as long as the formulas do not depend on the choice 
of the representative.) The induced action on a homology or cohomology class c is 
written as wc. 

Recall that the homology of X is torsion-free and has a basis given by the Schu- 
bert classes c w £ H 2 i( w )(X, Z), where w ranges over W and l(w) is the length of w. 
(The definition is reviewed in Appendix |a|) If wq is the longest Weyl group element, 
then Cw is equal to the fundamental class [X]. The homology being torsion- free, 
we can identify the cohomology with the dual abelian group Homz(H. (X, Z),Z). 
The cohomology class a w £ H 2l ( w )(X,Z) is defined by a w (c) = c • c WgW for c in 
H2i(w){X,Z) and a w (c) = if c has degree different from 21 (w). Recall that if 
l(w) + l{w') <l(wo), then 
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where the dot denotes the intersection product. This implies that the basis { o~ w \ 
w G W } of H'(X, Z) is dual to the basis { c w \ w G W } of H. (X, Z). 

Finally, the map <f> is the embedding of the flag variety X = K/T into the flag 
variety X = K/T which is induced by the map /: K — > K . ft is injective even if / 
is not. 

The Schubert bases depend on the choice of the dominant chambers. Ho wever , 
it is not hard to work out the formula for a change of basis (see e.g. Remark |A.l.l|) 
and to show that a different choice of compatible chambers t + and t+ leads up to 
a relabelling to the same set of inequalities. 



Theorem 3.1.1 makes the inequalities of the moment cone computable in practice 
once we can 

(i) determine the relative Weyl set W te \, 

(ii) write down inequalities for the codimension-one faces of the cones f*(vC), in 
other words, determine the rays that span their dual cones, 

(iii) write the action of the elements of Wrei on the homology in terms of the 
Schubert basis, 

(iv) find the nonzero entries of the matrix of 4>* with respect to the Schubert bases. 
Unfortunately, we do not know of a general solution to these combinatorial prob- 
lems except the third. (Formulas for the Weyl group action on the homology in 
terms of the Schubert basis were given by Bernstein et al. || and Demazure 
These are reviewed in Appendix ^|) In many interesting examples, however, they 
can be solved explicitly. Some properties that can be helpful in computing W rc \ 
and the cones f*(vC) are discussed in Section 2.2. Problem (|iv|) is discussed in 
Appendix [A]. 

3.2. Alternative formulations and corollaries. Let A G t+ and consider the 



coadjoint orbit 0\ = KX through A. It follows from (3.1) that the symplectic 
quotient ^~^}(0\)/K of T*K with respect to the right A-action is isomorphic, as 
a Hamiltonian A-manifold, to the dual orbit O- W0 \. Consequently the moment 
polytope A(O~ W0 \) of O~ wo \ with respect to the A-action is equal to the horizontal 
slice 

MO- lUo x) = A(T*K) n (t* x {A}) 

of the moment cone A(T* A). Furthermore, it is evident that A G A(0\) if and only 
if — w q\ G A(0~ Wq \). Thus, after substituting w — > wqw, we see that Theorem 
|3.1.l| is equivalent to the following statement. 

3.2.1. Theorem. Let (A, A) G t+ x t+. Then X G A(0\) if and only if 

w^Xe f*{w~ l \-vC) (3.5) 
for all triples (w,w,v) G W x W x W re l such that 4>*(va wv )(cw) / 0. □ 
Because of the equivalence of problems (|) and ([i]) discussed in the introduc- 



tion, Theorem 3.2.1 is tantamount to an asymptotic statement about irreducible 
representations. Let A = ker(exp| t ) be the integral lattice in t, A* = Homz(A, Z) 
the weight lattice and A!j_ = A* n tl the monoid of dominant weights. For every 
dominant weight A let V\ denote the irreducible A'- module with highest weight A. 

3.2.2. Theorem. Let (A, A) G h* + xA*^. Then there exists a positive integer n such 
that V \ occurs in V n \ if and only if (3.5) holds for all (w,w,v) G W x W x W re \ 



such that 4>* {va wv ){cw) ^ 0. □ 
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Similarly, it follows from (f3T|) that the symplectic quotient K\^^}(0) of T*K 

with respect to the left AT-action is symplectomorphic to the cotangent bundle of 
the homogeneous space K\K. Hence the moment cone of T*(K\K) with respect 
to the residual AT-action is equal to the "vertical slice" 

A(T*(K\K)) = A(T*K) n ({0} x t*) 

of the moment cone A(T*K). The following result is now immediate from Theorem 

3.2.3. Theorem. Let X e t* + . Then X G A(T*(K\K)) if and only if 

e f*{w- l \-vC) (3.6) 
for all pairs (w,v) G W x W te \ such that (jf(va wv ) ^ 0. □ 



Like Theorem 3.2.1, this has a representation-theoretic counterpart. 



3.2.4. Theorem. Let X G Ai_. Then there exists n > such that V n \ contains a 
K -invariant vector if and only if ( |3.6| ) holds for all (w, v) G W x W Te \ such that 
(f>*(va wv ) ^ 0. □ 



3.1.1, they are actually equivalent to it and play a part in the proof; see Section 



4.3 



Although we have presented Theorems 3.2.3 and 3.2.4 as corollaries of Theorem 



Suppose now that V\ occurs in V\. Then the weights in V\ are a subset of the 
weights of V\ (with respect to the torus T). The latter are of the form /*(A — 
fi), where fi is a combination of positive roots of K with nonnegative integral 
coefficients. In particular, A itself is of this form. From Theorem 3.2.2| we thus 
obtain the following necessary condition for V\ to contain V;. 

3.2.5. Theorem. Let (A, A) eA* x A+. IfV^ occurs in V\, then 

w^X G f*(w- x X-vC z ) 

for all triples (w,w,v) G W x W x W rc \ such that 4>*(va wv )(cw) ^ 0. □ 

Here we have written C% for the monoid generated by R+. An interesting question 
is for what pairs K this necessary condition is sufficient. 

3.2.6. Example. The answer to this question is clearly affirmative when K = T, 
the maximal torus of K. The same is true for the diagonal embedding of U(n) into 
U(n) x U(n), as was recently shown by Knutson and Tao However, it was 
pointed out to us by Knop that the answer is negative for the diagonal embedding 
G2 — > G2 x G2: the module V03 is contained in V30 ® V03, but Vbi is not contained 
in Via ® Vol- Here Vm is the irreducible module with highest weight kni + lirz, tti 
and 7T2 being the fundamental weights of G2. This can be checked using a result 
of Littelmann §3.8]. See Brion Q and Montagar jl9| for further references and 
related results. 



We discuss two more reformulations of Theorem 3.1.1. First, for all (w,w) G 



W x W let Cw, w be the cone |~L f*(vC), where the intersection is taken over all 



v G W re i such that (3.3) holds (and put Cw,w = i* if there is no such v). Theorem 



|3.1.1| is equivalent to: (A, A) G A(T*K) if and only if (A, A) G x t^ and 
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for all (w, w). 

Next, let P: W(X,Z) — > H 2 i( Wo )~i(X,Z) denote the Poincare duality map, 
V(a) = an [X]. Then V(a w ) = c WQW by @. Let 

! = V o 0* o V- 1 : Z) -► ff. +2(I(ao) _ I(t0o)) (1, Z) 

be the wrong-way or Gysin homomorphism induced by <f>. If l(w) + l(wv) — l(wo), 
then deg(j)*(v<T wv ) = degc^, so 

0*(wcr Ji , t ,)(c^ ot r,) = V((j)*(va wv )) ■ 

= ! (V(va wv )) ■ c WQW ^ 

= (-l)^ v '(fy(vVa wv ) ■ C Wo w 

( l) ^ ^ & ^C W Q WV ) • C W Q W . 

Here we have used that the mapping degree of the diffcomorphism of X induced by 



v is equal to (— l) l ^ v \ which follows from Theorem A. 2.2 . Let us say that a Schubert 
class c w is contained in a homology class c if it occurs with nonzero coefficient in 
the expression for c as a linear combination of Schubert classes. This amounts to 
o~w(c) 7^ 0, or equivalently, if c is homogeneous, c • c WoW ^ and degc = 2l(w). 



Then according to (p.7|), condition (3.3) is equivalent to 



c w is contained in 4>'{vc WQWV ). (3-8) 

This means that there exist cycles on X and X which are homologous to c w and 
vc WqWV , respectively, and which have a nontrivial transverse intersection. Applying 



V 1 to both sides yields a cohomological version of (3.8): 
a-waw is contained in 4>*{va wv ) 



Observe also that the condition 4>* {vcr wv )(c w ) of Theorem 3.2.1 is equivalent 
to 

a w is contained in <p*(va wv ). 



This reformulation of Theorem 3.2.1 does not refer to Poincare duality or the longest 
Weyl group elements and therefore makes sense, at least syntactically, for coadjoint 
orbits of certain infinite-dimensional groups. 



3.3. Scalar inequalities. Many of the vector inequalities ( |3.2| ) are redundant. In 
fact, each of them can be replaced by a single scalar inequality, and in addition 
( [3~3] ) can be reduced to a much smaller set of homological conditions. To do so we 
need to recall some more results from || . 

Let s be a (closed) face of the Weyl chamber t+ . The centralizer of s is equal to 
the centralizer of any point in its (relative) interior s° and is denoted by Zk (s) or 
K s . It is a connected subgroup of K and contains the maximal torus T. Its Weyl 
group Nk 3 (T) /T is denoted by W 5 , and the partial flag variety K/ K$ by X s . The 
homology of X s has a free basis consisting of Schubert classes c 5 w € H 2 i( w )(X 5 ,Z), 
where w ranges over the set W s defined by 

W* = { w e W I l(ww') > l(w) for all w' G W s }. (3.9) 

Every coset wW s contains a unique element in W*, called its shortest representa- 
tive. The canonical projection A — > A 5 induces a surjection H.(X, Z) — > H.(X S ,Z), 
which maps c w to c s w if w S W s and to if w W 5 . It also induces an embed- 
ding of H'(X S ,Z) onto the subspace of ^-invariants in H'(X,Z). A basis of 
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H*(X, Z) Ws is given by the Schubert cocycles { a w | w £ W 5 }. Via the identifica- 
tion H'(X S ,Z) = H'(X,Z) W ' this basis is dual to { c* w \ w E W* }. The Poincare 
dual of a w is c*, where u is the shortest representative of wqwW 5 . 

Consider any point x i n t+ an d let s be the face in t which contains x in its 
interior. Recall that t„ denotes the cone vt+ fit Choose u £ H^ ro i such that x £ t^, 
in other words v x £ t+, and let s be the face of t+ which contains v~ 1j x in its 
interior. Then clearly K% = vK s v f)K, so we can define an equivariant embedding 

fay.Xi^X, (3.10) 

by sending 7Tg(l) to n s (v). (Here 7r s denotes the quotient map K — ► X B .) Now select 
nonzero rational elements xi, X2, ■ ■ ■ , Xn 6 t such that each of the subcones U, of t+ 
is spanned by an appropriate subcollection. If trie = 0, this can be accomplished by 
selecting one nonzero element on every ray (one-dimensional face) occurring in the 



cubicle subdivision (|2.16| ). For k = 1, 2, . .., n, pick Vk £ W re i such that Xk £ Ui k - 
Let Sfe and Sfc be the faces of t and t such that Xk £ 5° k and v k Xk £ s&, respectively. 
For brevity put W 5k = W fc> Fy 5fc = W fc , and <fe fc: „ fc = fe . 

3.3.1. Theorem. Let (A, A) et^x t+. Then (A, A) £ A(T*K) if and only if 

(w- 1 X + r(w- 1 X),Xk)>0 

for k — 1, 2,..., n and /or (u;,k;) £ PF fc x W k such that dn k is contained 
in (<r Uk ) . Here Uk £ VF fe and Uk £ denote the shortest representatives of 
woivWk andwvkWk, respectively. 

For instance, if t+ is contained in t+ and K is semisimple, then we can choose 
the Xk to be the fundamental coweights. Then the cohomological conditions in 
this theorem involve only the minimal partial flag varieties ( "Grassmannians" ) of 
K and certain partial flag varieties of K, and hence the number of inequalities is 
considerably smaller than in Theorem |3.1.l| . In Sect ion || we shall see, however, 



that even the system of inequalities of Theorem 3.3.1 is often overdetermincd 



3.4. Duality. The momentum map on T*K has the property that $(fc, — £) = 
— $(fc, £). This implies immediately that A(T*K) is stable under the duality invo- 
lution, which sends (A, A) to (— woA, — wqX). Consider the inequality ( |3.2j ) associated 
with a triple (w, w, v) £ W x W x W Ie i- Substituting A — > — wqX and A — > — woA 
and multiplying both sides by wq we obtain 

woiu WqX £ — tt>o/*(w _:L i(7oA + vC). (3-11) 
This motivates the following definition. Let j : W <— ► W be the lifting homomor- 



phism associated with the chambers t+ and t+. (Cf. Remark |2.2.3| .) 

3.4.1. Definition. The dual of w £ W is w* — j(wo)wwo. The dual of w £ VF is 

W* = WqWUIq. 

Observe that (w*)* = w and (w*)* — w. However, the duality map on W is 
in general not a homomorphism, nor does it preserve the length function on W or 
map the subgroup W into itself. 

3.4.2. Lemma. The set of compatible Weyl group elements W com C W is stable 
under the duality map. Furthermore t W ' = —wot w and f*(v*C) — —wof*(vC) for 
w £ W com . Hence the subdivision into cubicles ( 2.16| ) is stable under the duality 
involution of t+ . 
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Proof. Using the definition of W CO m and the fact that u?ot+ = — 1+ we find 
w G W com wt+ n 

^ j(m)ww a t+ni° + ± 

W* G W C ora- 

By the same token, £ G t^* if and only if £ = j (wq)wwq£ i for some £ G t+, which 
is equivalent to — u)o£ = tu(— wo£) G totf PI t = t w . In other words, t«,« = — wqIu,. 



The equality f*(v*C) = —wof*(vC) now follows from Lemma 2.2.4, and the last 



statement is obvious. □ 



3.4.3. Example. Consider the map / : SU(3) 
By Q, 



U(15) studied in Example 2.3.2 



3 (wo) = j(h§2h) = (1 15)(2 13)(3 14)(4 10)(5 11)(6 12), 



and therefore 



< = (2 3 6 4) (7 9 12 10)(13 14), 
v* 2 = l* = (2 3) (4 6)(7 9)(10 12)(13 14). 



Comparing with (2.23) we see that the duality map does not map W Te \ into itself, 
but for each v G W re i, v* G Wv' for some v' G W le i- 



Together with ( 3 . 1 1[) Lemma |3 . 4 . 2 yields 



(3.12) 



which is called the dual inequality associated with ([T^). It can be obtained from 
( 3.2) by replacing the triple (w,w,v) with (to*, ((w~ 1 )*)~ 1 ,v*). We as sert that the 
dual inequality arises from a homological condition analogous to (|3.3|), namely 



0*(v*o-(( w -i).)-i v »)(ca o ts*) i 1 0. 



(3.13) 



Indeed, 



0*(w*<T(( Ji ,-i)»)-i 1 ,»)(c^ o ^») = 0*(j(*o)«U'OC r w)o^i«o)(cri)-!5o) 

= 0*(trtWo^u;oU>uwo)(woC^o) 

= (-l)'(^)+'(*0)-'(*)^( Wu ; fl )(c a0 *), 



where we have used the P T-eq ui variance of 6 (see Corollary |2.1.5| ) and Lemma |3.4.4 
below. This shows that ( |3.3| ) implies (3.13), which explains the appearance of the 
dual inequality (3.12). 



3.4.4. Lemma. w c w = (-1)'^^ 



and wqo- w = (— l) l ^a v 



Proof. The second equality is immediate from the first. By Remark A. 1.1, to prove 
the first equality it is enough to show that c™° = (— lY^'Cu,. We use the notation 
of that remark. The opposite or dual Borel subgroup is B w ° — wqBwq. Let 
9: G — > G be complex conjugation with respect to the real form K; then 9 maps 
B to its opposite and therefore induces an antiholomorphic map G/B — > G/B w °, 
which we also denote by 9. If Y is a complex submanifold of G/B, then so is 9(Y), 
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but 9 changes the orientation if the complex dimension of Y is odd. It is clear that 
the diagram 

X — — > G/B 



X — G/B w " 
commutes. Moreover, 9(X^ ) ) = (X™°)° and therefore 

(r W0 Uc w ) = e^{ Cw ) = e.([x w ]) = (-i) lw Kl = (-i)« w \t Wo )*c° , 

which shows that c™° = (-l) l ^c w . □ 

4. Semistability 



This section contains the proofs of Theorems |3 . 1 . l| and 3.3.1. In Sections 4.1 and 



4.2 we review the Hilbert-Mumford criterion and calculate Mumford's numerical 



measure of instability for flag varieties. We then finish the proof in Section 4.3 
The underlying idea is that an integral coadjoint orbit is a complex projective 
variety in a natural way, so we can detect points in its momentum polytope by 
determining the scmistable set with respect to various ample line bundles. This is 
achieved by means of the Hilbert-Mumford criterion, and the upshot is that the 
scmistable set is nonempty if and only if a certain translate of a Schubert cell in 
the flag variety X intersects the small flag variety X. By a trans versality argument 
this leads to inequalities in terms of intersections of Schubert cycles. 

Throughout this section the complexifications of the groups K and K are denoted 
by G and G, respectively. 

4.1. Measure of instability. The following material is extracted from Mumford's 
book pcfl . Let Y be a complex projective G- variety (where G — K c ) and let C be 
an ample G-equivariant line bundle over Y . Recall that y G Y is called semistable 
with respect to C if for some n > there exists an invariant section s of C n such 
that s(y) 7^ 0. A point is unstable if it is not semistable. The set of semistable 
points is denoted by Y ss . It is clearly Zariski-open and G-stablc. 

Let y be any point in Y and let x G Hom(C x , G) be an algebraic one-parameter 
subgroup of G. Mumford's numerical measure of instability is the integer m (y,x) 
determined as follows. Consider y$ — hhit_»o x(t)Uy which exists because Y is 
projective. It is a fixed point for the action of x and so x acts on the fibre £ yo . Let 
r be the unique integer such that x(i) I = VI for I € C Vo . Then m c (y, x) — r - (This 
sign convention is opposite to Mumford's, but agrees with our sign convention for 
the moment map.) 

4.1.1. Theorem (Hilbert-Mumford criterion). A point y GY is semistable if and 
only ifm c (y,x) ^ f or a ^ one-parameter subgroups X- 

We will frequently use the following elementary properties of m c (y,x)- 

4.1.2. Proposition. (i) m c (gy,gxg^ 1 ) = rri c (y,x) for all g E G. 
(ii) m c (py,x) = m c (y,x) for all p G P x . 

(hi) If(Yi,C\) and (Y^,-^) are two linearized G '-varieties ■, then 



CimC2 ((2/1, ya), x) = ™ Cl (yux) + ™ C2 (2/2, x) 



for (yi,y 2 ) eY\XYq 



2- 
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Here P x denotes the parabolic subgroup associated with x> which is defined 
in (A..1). We identify Hom(C x ,H), the set of algebraic one-parameter subgroups 
of the complex Cartan H = T c , with the lattice A C t by assigning to each 
one-parameter subgroup x its infinitesimal generator dx(l) times 2-ki. We call 
X dominant if it is contained in A + = A n t+ and denote by (£, x) the natural 
pairing between x and any £ 6 g*. According to 4.1.2(j), to calculate m c (y,x) 
for arbitrary one-parameter subgroups x of G it suffices to calculate m (gy,x) f° r 
arbitrary g G G and dominant one-parameter subgroups x of H. 

4.2. Instability on flag varieties. The following discussion generalizes §4.4]. 
Take Y to be the flag variety X — K/T . We identify X with the complex homoge- 
neous space G/B and consider the homogeneous line bundle 

C x = Gx B C x , 

where A is a strictly dominant weight and Ca is the one-dimensional representation 
of T defined by the weight A (extended to B by letting H — T c act holomorphically 
and by letting the unipotent radical of B act trivially). Instead of m Cx we write 
m x . Consider an arbitrary x G X. Let it: G — » X be the quotient map and write 
x = 7r(<?). We assert that if h G G and x is dominant, 

m x {x,h X h- 1 )=m x (h- 1 ir(g),x) = (ti^A,*), (4.1) 

where w is the unique Weyl group element for which 7r(/i) is in the translated cell 
<?X° . Here we use the notation X£, for the Bruhat cell BwB / B C X corresponding 
to w G W. Indeed, n(h) G gA^, is equivalent to h~ 1 n(g) being in the B-orbit of 
7r(w _1 ). Since X is dominant, P x contains B, so 

m x (hr 1 ^{g),x) =m x (ir(w~ 1 ) lX ) 

by 4.1. 2| (P). Because tt(w~ 1 ) is fixed under H, we can calculate the right-hand side 
by considering the action of X 011 the fibre of C\ = G x B Ca over 7r(w _1 ): 

X(t)[w-\z] = [x(t)w-\z] = [w-'^wx^w-^z] = [w-\t^- lx ^z\. 

This proves m A (7r(w _1 ), x) = (w _1 A,x) and hence ( |4.l|) . 

Evidently, there are no G-semistable points on A, but the situation becomes 
interesting when we restrict the action of G to the subgroup G — K . Again we need 
only calculate m (x, gX9~ 1 ) — m \9 ~ l ^{9)iX) f° r arbitrary g G G and dominant 
X G Hom(C x , H). Now x is not necessarily dominant for G, but according to ( [2.16 ) 
we can choose v G W rc \ such that w _1 x is dominant for G. Then m A (<7 -1 7r((7), x) — 



in 



x ^v 1 g 1 ir(g),v 1 x)i which can be computed by (|4.l|). The upshot is 



m x (x,gxg- 1 ) = m x {g- 1 <9),x) = {fiw^X)^), (4.2) 

where w G W is determined by the condition that gv G gBwvB, i.e. gir{v) G jZ°„. 
By Proposition 2.2.6| (|v|) the stabilizer of ir(v) G A with respect to the G-action is 
the Borel B. Let <p v : X — > A be the G-equivariant embedding which sends 7r(l) to 
7r(i>). Using the Hilbcrt-Mumford criterion we obtain the following result. 

4.2.1. Proposition. Let X be a strictly dominant weight. A point Tr(g) in X = 
G/B is semistable with respect to the bundle C\ and the action of the subgroup G 
if and only if 

(r(w- 1 X),x)<0 (4.3) 
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for all {w,v) £ W x W ve \ and xe A such that x £ A + n vA + and gX^ v n<f) v (X) is 
nonempty. □ 

If A is dominant but not strictly dominant, then it does not define an ample 
bundle on X, but on the partial flag variety X s = G/P, where s is the face of t+ 
which contai ns A in its interior, and P = P s , the parabolic associated with s. The 
analogue of (42) is: for all g and dominant x 

m A (r 1 7r,(ff),x) =(/*(* 



L A),x), 



where u> is any element of W (unique modulo left multiplication by We) such that 
gv £ gPwvB, i.e. 0w (7r(<jf)) £ g{P^(wv)) . (Note: this conditio n invo lves a P-orbit 
in G/P, not a P-orbit in G/P.) By analogy with Proposition 4.2. 1 we obtain the 
following. 

4.2.2. Proposition. Let A 6 A* fls°. A point ir 5 (g) in X s = G/P is semistable 
with respect to the bundle C\ and the action of the subgroup G if and only if ( |4.3|) 
holds for all (w, v ) £ W x W rc i and \ £ A such that \ £ A+ nuA+ and g(PTr(wv)) fl 
zs nonempty. □ 



It is in fact not necessary to verify (4.3) for all dominant one-parameter sub- 
groups of H. Choose a collection of dominant one-parameter subgroups x%, x%i ■ ■ ■ > 
X n of P such that each of the cubicles t v in t + is spanned by an appropriate sub- 
collection. Then every dominant one-parameter subgroup x can be written as 



1 



X 



k=l 



a-kXk, 



(4.4) 



where a and afc are integers, a > 0, at > 0, and = if Xfc is not in the cubicle 



containing %. Now let A G A^ be strictly dominant. It follows from (4.2) and ( |4.4| ) 
that ir(g) G X is semistable for the action of G if and only if m x [g~ 1 Tr(g), Xk) < 
for all fc and for all g G G. For fc = 1, 2, . . . , n, fix uj, £ Wr C i such that Xfe £ t„ fc 
(i.e. Xk is dominant relative to G) and let resp. Sfc, be the face in t+, resp. 
tf, which is determined by Xfe £ 3£, resp. f -1 ^*; £ s^. For simplicity let us write 

' rk 



X k = X Sk , P k = P Sk , W Sk = W k , W s " = W\ and n ik = % k . Let 
be the embedding of X k into X k defined in ( 3.1 0| ) . Again using 4.1.2 
stronger version of (4.2): 



we get a 



m x {g- 1 Ag),Xk) = {r{w- 1 \),xu), 

£ W k is determined by gv k £ gB>wv k P k , in other words (f>k(^k(g)) £ 
g(Xk)wv- ■ The conclusion is as follows. 



where w 
)° 



4.2.3. Proposition. Let X be a strictly dominant weight. A point n(g) in X is 
semistable with respect to the bundle C\ and the action of the subgroup G if and 
only if 

(f*(w- 1 X),x k )<0 

for k = 1, 2, . . . , n and for all w £ W k such that g(X k )^ k n (j> k (X k ) is nonempty. 
Here u k £ W k denotes the shortest representative of wv k W k . □ 



We leave it to the reader to state a version for singular dominant weights. 
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4.3. Proof of Theorems and |3.3.l| . The proof of Theore m |3.1.1 



two steps: first we show that Theorem 3.2.4 implies Theorem 3.2.2 , and then we 
establish Theorem 3.2.2. As we pointed out earlier, Theorem 3.2.2 is equivalent to 
Theorem |3.1.l| , so this will finish the proof. 

Step 1. Suppose we knew Theorem 3.2.4 was true. Let us apply it to the pair 
(K,K x K) and the inclusion map id^- xf. Consider (A, A) G A!j_ x A!j_ and note 

that (Vr <S> V\) K ^ {0} if and only if V\ is a summand of V£ = V- WQ \. Hence, by 
Theorem |3.2.3| , there exists n > such that V n % occurs in V- nwo \ if and only if 



x/)*((tS- 1 A,«;- 1 A) 



G (id* 
for all (w,w,v) such that 



CxvC) 



(4.5) 



(4.6) 



Now (id* xf)*(C x vC) = C + f*(vC) = f*(vC) by Proposition |2.2.6 
equivalent to 

^X + vC). 



w- 1 



Ag/*(- 



■w 



Furthermore, (L6) is equivalent to 



so (4.5) is 
(4.7) 

(4.8) 



in H2m(X, Z), where m = 1(ivq) — 1(w)—1(wv). The conclusion is that V i Q V- nW[> \ 
for so me n > if and only if (17) holds for all w, w and v satisfying ( |4.8|) . 

In ( |4.8| ) it is actually sufficient to take into account only those triples (w,w,v) 
for which the intersection is zero-dimensional. To see why, recall that the class c^ ^ 
is represented by the Schubert variety X^^ ^= X and that vc WQWV is represented 
by an algebraic cycle X w _ v which is a linear combination of Schubert varieties in X. 
Then (4.8) means that for generic g G K and g G K the translated cycles gXw w 
and gX w v intersect in an m-dimensional cycle on X. If m > 0, this implies they 
intersect in a boundary component X!~ fi of X^^ because of the fact that X!j~ - 
is an affine variety. In other words, we have Cw u ' (vCw oWV ) ^ for some u )p w 
(where >p denotes the Bruhat-Chevalley order), and hence 

u- x X G n-w^X + vC). (4.9) 

Because A is dominant, {t _1 A < ui _1 A, so if (4.9) holds, then 

w^X G u _1 A + C C /*(-tw- 1 A + vC) +C = n-w^X + vC). 



This shows that the inequality (4.9) is stronger than (4.7). By induction we can 

except those for which m = 0, in which case they are 
«>o«0 7^ 0. Thus we see (after replacing A by — wqX and w 



discard all inequalities (4.- 
equivalent to (f>*(va v 



>)(c 



by Wqw) that Theorem 3.2.4 implies Theorem 3.2.2 



Step 2. We now prove Theorem 3.2.4. Consider A G A3_. Let us assume first that 
A is strictly dominant. Le t C\ the homogeneous ample line bundle on X = G/B 
introduced in Section 4.2 . The Borel-Weil Theorem says that the space of holo- 
morphic sections of C 7 ^ is isomorphic to V n \ as a if-module. Hence, by definition, 
there exists n > such th at V n \ contains an invariant vector if and only if X ss is 
nonempty. By Proposition 4.2.1 this is equivalent to the existence of a g G G such 
that 



G f*{w- l \ + vC) 



(4.10) 



24 



ARKADY BERENSTEIN AND REYER SJAMAAR 



for all (w, v) G W x W le \ satisfying 



(4.11) 



Here we have used that f*(vC) is dual to the cone spanned by A + n vA + , which 



follows from Lemma [2.2.4 . Since the semistable set is Zariski-open, by Kleiman's 
transversality theorem (see e.g. Theorem 10.8 in |ll], Ch. Ill]) we can perturb g to 
move gX^ v into general position with respect to <f> v (X). Then (4.11) amounts to 
(p'yicwv) 7^ 0. Now observe that the holomorphic embedding <j> v : X — > X is equal 
to the composition v^ 1 o (ft (even though v^ 1 is not holomorphic). Hence 

? <ft[{c wv ) = (v- 1 o cft)-(c wv ) = cft\v^)\c wv ) = (-l) l( - v U'-{vc wv ). (4.12) 

Substituting A — > — WqA in (4.10), it 
V n x contains a If -invariant vector if and only if V*^ = 



Wqw in (4.1C) and ( 4.12] ), and using that 
a does, we obtain 
Theorem 3.2.4 for strictly dominant integral A. 

The case where A 6 A^ is dominant but not strictly dominant is very similar. 
We briefly point out the main differences. Instead of X we use the partial flag 
variety X s = K/K s = G/P with the G-equivariant line bundle C\ = G x p C\. 
Here s is the face of t + such that A e s°, P = P s is the parabolic subgroup of G 
attached to s, and C\ is the obvious P-module defined by A. For any w E W the 
closure of the P-orbit through n(w) is a single Schubert variety, namely X w t, where 
w' is the longest element in the coset W s w C W . The same reasoning as above 
(using Proposition 4.2.2] instead of 4.2.1) now gives that A G A(T*(K\K)^J if and 
only if (4.10) holds for all w and v satisfying 

4> l (vc Wv ) ? 0. (4.13) 



But if (4.10) holds for w, it holds for all elements in the coset W$w, because W s 
fixes A. This finishes the proof of Theorem 3.2.4. 

The proof of Theorem 3.3.1 is almost exactly the same, but uses Proposition 
4.2.3] instead of 4.2.1. The details are left to the reader. 



5. Examples 



5.1. Kostant's theorem. As a simple illustration of Theorem 3.2.1 we consider 
K = T, the maximal torus of K . The answer here is of course well-known and can be 
obtained in many other ways. Observe that T = T, W = {1}, and every element 
of t is "dominant" with respect to T. Therefore W rc \ = W, t„ = ut+., and the 



decomposition ( 2.16| ) is simply the partition of t into Weyl chambers, t = [j v 



Vt 



+ ■ 



The dual cone to vt+ is vC, where C is the root cone of K. The flag variety X is a 
point, so (ft* is the trivial homomorphism H'(X, Z) — * Z. Since (ft* preserves degree, 
we see that (ft*(va wv ) ^ if and only if v = w . In other words, for every A € ti 
the points A in the polytope A(0\) are described by the inequalities wX E A — C, 
where w ranges over W. This is equivalent to Kostant's result that A(0\) is the 
convex hull of the Weyl group orbit WX. 

5.2. Klyachko's theorem. In this section we apply our main theorem to the 
diagonal embedding of a group into a number of copies of itself. We deviate from 
our standard notation and denote the small group by K and the large group K x 
K x ■ • • x K (m times) by K m . The canonical projection /* : (t*) m — > t* i s the 
addition map. There is a canonical embedding of the small Weyl group W into the 
large Weyl group W m , namely the diagonal embedding. This tallies with Example 
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2.1.6] , because every regular point in the diagonal subgroup K <^-> K m is regular 



relative to K m . For the same reason, W Te \ = {1} and the partition (2.16) has only 
one piece, namely ti = t+. Its dual cone is C, the root cone of K. Moreover, the 
large flag variety is X m and the embedding <j>: X — > X m is the diagonal map. The 
induced homomorphism (f>* : H'(X,Z)® m S H'(X m ,Z) -> H'(X,Z) is the cup 
product, and the Gysin map 

4>- : iJ 4 (X" l ,Z) — > Hj_( m _ 1 ) J ( TOo )(X,Z) 



is the intersection product. Similarly, for every face s of t + the embedding ( |3.10 ) 



is simply the diagonal embedding of the partial flag variety X s into the m-fold 
product X" L . Now choose a collection of vectors Xii X2, ■ ■ • > Xn S t which span the 



cone i + . Then Theorem 3.3.1 comes down to the following 



5.2.1. Theorem. Let (A 1; A 2 , . . . , A„ l+1 ) 6 (t;)" l+1 . Then (A x , A 2) . . . , A m+1 ) £ 
A((T*K) m ) if and only if 



m+l 

X>r%X*>>0 (5.1) 
1=1 



for k = 1, 2, . . . , n and /or aZZ (z«i, w 2 , • ■ ■ , Wm+i) G W k x FF* x ■ • • x VF fc such 
that a Uk is contained in a wi U a W2 U • • ■ U <J Wm ■ Here itf. £ W k denotes the shortest 
representative of WoW m +iWk- 

For K = U(n) this theorem was proved by Klyachko p5[. Also for K = U(n), 



Helmke and Rosenthal 1 1 3 proved that the inequalities (5.1) are necessary, but 
not that they are sufficient. Inequalities for eigenvalues of sums of Hermitian ma- 
trices have a long history going back to Weyl; see the cited papers for examples. 
The essential case of the theorem is of course when K is semisimple; the identity 
component of the centre just contributes a number of equalities, e.g. for the trace 
when K = U(n). The cup product condition is closely related to the Littlewood- 
Richardson rules, which are discussed in Fulton ^j, Pragacz |2lJ and Littelmann 

5.2.2. Example (G 2 x G 2 ). Let us workout the case K = G 2 and m = 2. Denote 
by S = {ai,a 2 } the simple roots of G 2 , where ot\ is short and a 2 is long. Let 
S\ = s ai and s 2 = s a2 be the corresponding simple reflections and tt\ = 2a\ + a 2 
and 7r 2 = 3ai + 2a 2 the fundamental weights. For k = 1 or 2, let Pk be the parabolic 
attached to it/,, Wk the Weyl group, W k the set of shortest representatives for 
W/Wk, and X^ the associated ten-dimensional "Grassmannian" . Then 

W\ = {1, s 2 }, W 1 = {l, si, s 2 si, sis 2 si, (s 2 si) 2 , si(s 2 si) 2 }, 

W 2 = {1, Si}, W 2 = {l, S 2 , SiS 2 , s 2 sis 2 , (sis 2 ) 2 , s 2 (sis 2 ) 2 }. 



Using Chevalley's formula, Theorem |A . 2 . 1 ( iij) , we can compute multiplication ta- 



bles for the cohomology of Xk in terms of the Schubert bases. We find the relations 

cr s 1 = cr s2si> °si = 2o- slS3S1 , 0-^=2(7(^^)2, o- Sl = 2o- Sl(g2Sl )2 for fc = 1, 
o" 2 2 = 30-SJS2, er S2 = 6ct S2S1S2 , o-^ 2 = 18cr( SlS2 )2, cf 2 = 18o- S2 ( SlS2) 2 for fc = 2, 
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from which it is easy to derive all triples (u>i, w 2 , w 3 ) G W x W k x W such that 
a Uk is contained in a Wl U a W2 . For k = 1 they are: 

(l, 1, si(s 2 si) 2 ), (l, si, (s 2 si) 2 ), (1, S2S1, SlS 2 Si), 
(si, Sl, S1S2S1), (Si, S2S1, S2S1) 

plus permutations of these triples. For fc = 2 we obtain the same list, but with 
si and s 2 interchanged. If we identify q 2 with q 2 by means of the invariant inner 
product for which ||ai|| 2 = 2, then we can choose the Xfc to be the fundamental 
weights 7Tfe. The inequalities for the polygons A((D\ 1 x 0\ 2 ) are best written out 
in coordinates relative to the basis {7ri,7T2}, because then the positive chamber is 
given by the positive quadrant. Writing A, = Xiiri + yi~K2, we get for k = 1 

2xi + 3yi + 2x 2 + 3y 2 - 2x 3 - 3y 3 > 
2xi + 3j/i + x 2 + 3y 2 - x 3 - 3y 3 > 
2a;i + 3j/i + x 2 — x 3 > 
a^i + 3yi + x 2 + 3y 2 - x 3 > 
xi + 3yi + x 2 + x 3 > 

and for k = 2 

xi + 2yi + x 2 + 2y 2 - x 3 - 2y 3 > 
xi + 2yi + x 2 + y 2 - x 3 - y 3 > 
xi + 2yi + y 2 - y 3 > 
x\ + yx + x 2 + y 2 - y 3 > 
xi + yi + yi + y 3 > 0, 

up to permutations of the variables. The triples (si, s 2 si, s 2 si) and (s 2 , sis 2 , sis 2 ) 
(the associated cycles of which have intersection multiplicity > 1) turn out to be 
redundant. The inequalities associated with the triples containing a 1 express the 
fact that 

A(0 Al x C>a 2 ) C (Ai +hullVFA 2 ) n (A 2 +hullVFAi). 

Figure |^ shows an example where this inclusion is strict. The dark shading indicates 
the moment polygon and the light shading its Weyl group translates. The dotted 
lines are the polygons Ai + hull WX 2 and A 2 + hull WXi. 

5.3. st(2)-triples. Our next example is a general homomorphism with finite ker- 
nel f:K—>K, where K = SU(2) and K is semisimple. Up to conjugacy such 
homomorphisms are in one-to-one correspondence with embeddings of § = st(2,C) 
into g, that is to say, triples (h, e, /) of vectors in g which satisfy 

[h,e]=2e, [hj]=-2f, [ej]=h. 

Indeed, because SL(2, C) is simply connected any such embedding lifts to a homo- 
morphism SL(2,C) — > G, which can be conjugated to a homomorphism that maps 
SU(2) into K, since K is maximal compact in G. In other words, we may assume 
that h G t, e G n and / G On. After a further conjugation with an element of W we 
may even assume that h G t+. Then the chambers t+ and t+ are compatible and 
W iel = {!}. 
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FIGURE 3. Moment "rosette" of 0\ 1 x 0\ 2 , where K = G 2 , Ai = 
i(57Ti + 7r 2 ), and A 2 = \{-k\ + 37r 2 ) 

Let {a* I a G S } be the set of fundamental coweights, i.e. the basis of t which 
is dual to S C t* . Then we can write 

aeS\S 

where all d a > and £ C S is the set of simple roots orthogonal to h. According 
to Dynkin's classification of sl(2)-triples (see 0), d a — 1 or 2 for all a $ S. The 
set S is a base of the root system R oi K — Z K (T), and W is generated by the 
simple reflections s a with a 6 S. Let j : W = {1,wq} ^ W be the inclusion map 
determined by the choice of dominant chambers t+ and t+. This map is easy to 
describe explicitly. 

5.3.1. Lemma. (i) woh = —h. 

(ii) j(wo) = wqWo. 
(hi) WoWq is an involution. 

Proof. The W-equivariance of the embedding /* : t — > t implies that j(wo)h = —h, 
which is in the antidominant chamber — 1+. On the other hand Woh is also in — 1+ 
and hence u>oh — —h. This proves (|). 

To prove (0) it suffices to show that WqWo is in Afw(S) an( i maps h to —h. It 
follows from |[) that wo preserves R, and therefore sends R+ to R- . Consequently 
wowo preserves S. Since W hxes h, wowoh = woh = —h by (§). 

(|iii|) follows immediately from ([|). □ 

In order to write the inequalities for the "polytopes" A (0a) we identify t* with 
M by sending the positive root a to 2. Dually, this corresponds to sending the basis 
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element ft G t to 1. The projection /*(A) G t* of any functional A £ t* then gets 
identified with the number A(ft). 

5.3.2. Proposition. Let A e tl_. Then A 6 A(0\) if and only if A > and 
-A(ft) + max d Q A(a v ) < A < A(ft). 

a£S\S 



Proof. Following Theorem ft. 2.1 we determine all (w,w) £ W x W such that 



4>*{^ w ){cw) ¥= 0, i.e. a*, is contained in (f>*(a w ). Since X = CP 1 , H 2l ^(X,Z) 
vanishes for l(w) > 1. Moreover, <jf preserves degree, so that we need only consider 
Weyl group elements of length < 1. 

For w = w = 1 we find = a w = 1 and get the inequality A < A(ft). 
If l(w) — l(w) — 1 then w = wq and w = s a for some a 6 S. Using Theorem 



A.2.1 we find that <fi*(o~ Sa ) — f*{^a)o^w — ^a(h)aw , where n a denotes the fun- 



damental weight corresponding to a. Therefore dw Q is contained in <p*(a Sa ) if and 



only if ir a (h) ^ 0. From (5.2) we obtain 

7T a (ft) = dpTT a ((3*). 

f3eS\5 

Since the angle between any two fundamental weights n a and np is acute, we have 
7i"a(/3*) > for all a and f3, and therefore ir a (h) ^ if and only if ir a ((3*) ^ for 
some (3 S S \ S. This is certainly the case if a € S \ S, because then we can take 
(3 = a, but may also happen if a G S. In any case, the inequality corresponding to 
(w ,s a ) is w X < f*{s a h) = A(s Q A), or 

A > -A(s Q ft) = -A(ft) + X(a v )a(h). 

Here we have used that wqX = —A and s a X = X — A(a v )a. Moreover, 



a(h)= d P a (P) 
pes\s 



d a if a £ S 
if a G S. 



Thus, for a G S we obtain the inequality A > — A(ft), which is vacuous, and for 
a g S we obtain A > -A(ft) + X(a w )d a . □ 

5.3.3. Example. There exists a triple (ft, e, /), known as the principal triple, 
for which 5 = and d a — 2 for all a. It was shown by Dynkin that prin- 
cipal triples are unique up to conjugation. In this case we get the inequalities 
—X(h) + 2max a£ s A(a v ) < A < X(h). For instance, if K is the product of m 
copies of SU(2), then the principal SU(2) is the diagonal subgroup. Here A can be 
represented as an m-tuple (Ai, A2, . . . , A m ) and the inequalities are 

— Ai — A2 — • • • — A m + 2 max Ai < A < Ai + A2 + • • • + A m . 



This can of course also be regarded as a special case of Theorem 5.2.1 



5.4. A maximal rank subgroup. Let K — G2. The long roots form a subsystem 
of the root system of K and the associated subgroup K is isomorphic to SU(3). 



Using the notation of Example 5.2.2, we denote the simple roots of K by a.\ and 



«2 and its fundamental weights by 7Ti and 1x2- Writing u\ and 612 for the simple 
roots of K, and %\ and n 2 for its fundamental weights, we have ot\ = \{ai — a 2 ), 
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FIGURE 4. Moment rosettes of G 2 -orbits 0\ w.r.t. SU(3)-action, 
where A = tt\, resp. A = 7r 2 (complexified adjoint representation) 



<^2 = Ot-2, n l = K\ and 7T 2 = TTl 

weights is therefore 



7T2- The matrix of /* relative to the fundamental 



r = 



i i 

1 



(5.3) 



The dominant chamber of K consists of t wo ch ambers of K, and the relative 
Weyl set is {1, si}. According to Theorem 3.3.1 we need to consider three one- 
parameter subgroups xii X2) X3j an d corresponding embeddings 4>\\ G/P\ — ► G/P%, 
(j>2- G/B — » G/P2 and 4>3- G/P2 — > GjP\. However, the inequalities coming from 
</>3 are dual to those of <fii , so we need to consider only </>i and (f>2 • The cohomology 



of the Grassmannians of K was computed in Example 5.2.2, The calculation for 



K is very similar. By means of (5.3) and Theorem A. 2.1 we can then calculate all 
pairs (w,w) 6 W k x W k such that a^i is contained in <j>l(u w ). For k = 1 we find 

(1,1), (S1,S%), (S2S 1 ,S 2 S 1 ), 

and for k = 2, 

(1,1), (si,s 2 ), (s 2 ,s 2 ), (5lS 2 ,SiS 2 ), (s 2 Si,SiS 2 ), (w ,S2SlS 2 ). 

In fact, for all these pairs occurs with multiplicity 1 in 4>%(crw) ■ Introducing 
coordinates A = xtti + y7r 2 and A = xtti + j/7r 2 , we can then write the inequalities 
(?i _1 A,Xfc) < {w~ 1 \,Xk) as follows. It turns out that the inequalities for k = 1 are 
all redundant, as are their dual inequalities (which are obtained by interchanging 
x and y). For k = 2 the pairs (sis 2 , sis 2 ) and (s 2 si, sis 2 ) l ea d to redundant 
inequalities. The remaining ones are 

x + y < x + 2y, x < x + y, y < x + y, x + y > y. 

This set of inequalities is self-dual (stable under interchanging x and y) . In addition 
to the obvious inequalities x > 0, y > 0, x > 0, y > 0, these completely describe 
the set of all (A, A) e x such that A e A(C A )- See Figure || for examples. 
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Appendix A. Flag varieties 

A.l. Schubert cells. Let G = K c be the complexification of A and H = T C the 
complexified maximal torus. Let R C t* be the root system of A and R+ the set 
of positive roots. Let n be the nilpotent subalgebra of g spanned by the positive 
root spaces and N = expn the corresponding maximal unipotent subgroup of G. 
Furthermore, let b be the Borel subalgebra f) © n and B — exp b the corresponding 
Borel subgroup. The canonical map 

t: X = K/T -> G/B 

defined by r{kT/T) = kB/B is a A-cquivariant diffcomorphism. The complex ho- 
mogeneous space G/B decomposes into Bruhat cells 1° = BwB/B, where w G W. 
The Schubert variety X w is the closure of X^ ; the Schubert class is the fundamental 
class [X w ] E H 2 i( w )(G/B,Z), where l(w) is the length of w. Define 

c w = t~ ([X w ]); 

these classes form the Schubert basis of the homology group H.(X, Z). 

A. 1.1. Remark. The Schubert basis of H,(X, Z) depends on the choice of the set 
of positive roots. Let us work out the formula for a change of basis. Any set of 
positive roots can be written as uR + for a unique u £ W, and the Borel associated 
with uR + is B u = uBu' 1 . Let us write {X™)° for the Schubert cell B u wB u /B u in 
G/B u , t u for the canonical A-equivariant diffeomorphism X — > G/B u 1 and for 
the homology class (tu)" 1 ([X^]) . Consider the commutative diagram 

X G/B 



X Tu ) G/B u , 

where the vertical arrow on the left denotes the action of u on X and the map ip u 
is the G-equivariant holomorphic map ip u (gB/B) — gu~ 1 B u / B u . It is clear that 

iK) = u-'PCn-i) , so we see that 

{t u )*{uc w ) = Ou)*T«( Ctu ) = (ip u )*([X w ]) = [X^ wu -i] = (r„)*c^„_i. 

We conclude that the Schubert basis relative to the set of positive roots uR+ is 
given by c™ = uc u -i wu . 

If P is any parabolic subgroup of G, we have the partial flag variety G/P and 
a A-equivariant diffeomorphism r: K/(P n A) — > G/P. We call P standard if 
P D B; then P = G S N, where G 5 is the complexification of K s = Zk(s), the 
centralizer of a face s C t + . In this case we write P — P s and A/A s = X s . Every 
parabolic is of the form 

P x = {peG \ limx(i)px(i) _1 exists} (A.l) 

for some algebraic one-parameter subgroup x £ Hom(C x , G), and we have P x = P s 
if and only if (the infinitesimal generator of) \ is in s°. The Schubert cells in G/P 
are the sets (A s )^ = BwP/P; their closures are the Schubert varieties (X s ) w , and 
the classes = T„r 1 ([(A s ) tu ]) form the Schubert basis of i?.(A s ,Z). Here w ranges 
over the subset W s of W. 
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A. 2. Cohomology. Consider the map H'(X,Z) — > H'(X,Z) induced by the em- 
bedding 4>: X -> X. Note that X = [K, K]/(T n [K, K}) and that / maps [K,K] 
into \K, K] . Hence <f> can also be viewed as the embedding induced by the homo- 
morphism [K, K] — > [if, if]. For the purpose of this discussion we may therefore 
assume both K and K to be semisimple. 

Define a map 9: A* — > H 2 (X,Z) by 8(A) = c\(C\), the first Chern class of 
the homogeneous line bundle with weight A. See M or || for the theorems quoted 
below. 

A. 2.1. Theorem (Chevalley). (i) is an isomorphism. 

(ii) 0(7r Q ) = cr Sa for all simple roots a. 

(iii) For all weights A 

l(wsp)= l(w)-\-l 

Parts (@) and (Q lead to the following simple description of the map <jf restricted 
to H 2 . It is clear that the diagram 

A* - > A* 



H 2 (X,Z) — £— » H 2 (X,Z) 

commutes. In degree 2, the matrix of <jf relative to the Schubert bases is therefore 
the same as the matrix of /* relative to the bases of fundamental weights. 

Now consider the graded algebra S(t*) of polynomial functions on t (in which 
elements of t* arc defined to be of degree 2) and the ideal J generated by the 
W-Mnvariant polynomials of positive degree. Recall the following well-known result. 

A. 2. 2. Theorem (Borel). The map extends to a surjective homomorphism of 
graded algebras S{i*) — > H*(X,M), whose kernel is equal to J. 

Observe that f*(J) C J, so that the map cf>* can be alternatively described 
(at least over R) as the homomorphism induced by the restriction map 5(t*) — » 
5(t*). Thus the degree-2 Schubert classes a Sa generate the cohomology of X over K 



(though not over Z), and by using Theorem A.2.1(iii) one can in principle calculate 
the matrix of 4>* m higher degrees. This can be laborious in practice, however. 

A convenient method for calculating the Weyl group action on the cohomology 
is provided by the difference operators D a : 5(1*) — » 5(t*), which are defined for 
ae Rby 

D a (p) = 

a 

where p G 5(t*). It is clear that D a (J) = {0}. Hence, by Theorem A.2.2 , D a 
induces a linear operator of degree —2 on H*(X, R), which will also be denoted by 
D a . Note that D 2 a = for all a. 

A. 2. 3. Theorem (Bernstein et al. ||, Demazure ||). For all simple roots a and 
all w E W we have 

if ws a y w 
<r WSa if ws a -< w. 



D a {o- w ) 
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Thus s a a w = u w — Q(a)D a (a w ), which is equal to a w if ws a y w and to 
(T — 6(a) U <r WSa if ws a ~< w. 

Appendix B. Notation 

AT; T; G compact connected Lie group; maximal torus; centre 

G; H complcxification of AT; resp. T 

W; wq Weyl group of K w.r.t. T; longest Weyl group element 

R; R + root system; positive roots 

S; C simple roots; cone spanned by S 

N; B maximal unipotent subgroup of G; Borel subgroup HN 

Afj(J); Zj(J) normalizer; resp. centralizer of group J in a group I 

AT; 7T full flag variety K/T = G/B; projection G -> X 

t+; t+ positive Weyl chamber in t; positive Weyl chamber in t* 

A; A* integral lattice in t; weight lattice Homz(A,Z) 

A + ; A!j_ dominant one-parameter subgroup; dominant characters 

0\ coadjoint orbit through Aet* 

/; /* homomorphism K — > AT; dual projection fi* — > I* 

AT°; X w Bruhat cell BwB/B C A; its closure 

c w ; a w Schubert cycle [X w ] in H 2 u w )(X,Z); cocycle in H 2L ^(X,Z) 

V Poincare duality H l (X, Z) -> H 2l(wa) ^(X, Z) 

0; ! embedding A — > A"; wrong-way map V o 0* o "P -1 

s; s° face of t + ; its relative interior 

AT S ; i? 5 centralizer of s; its root system 

W s ; W s Weyl group of K s ; set of shortest representatives of W/W s 

P s ; X s parabolic G 5 N; partial flag variety K/K s = G/P s 

ir s ; (pi projection G — ► A' s ; inclusion Ag — > A s (if Aj C K s ) 

C x line bundle G x Ps C\ on AT 6 (for A G s° n A*) 

P x parabolic associated with one-parameter subgroup x of G 

(-Xs)° ; (X s )„, Bruhat cell BwP/P C A~ s ; its closure 

Schubert cycle [(A' 5 ) u) ] in H 2 i( w )(X s ,Z) (for w; G W s ) 

t w ; W rc i cone wt+ D I (for u> G W) ; relative Weyl set 

Similar conventions are in force for the groups K and K, i.e. G and G denote the 
centres of K and A', G and G their complexifications, etc. We assume furthermore 
that f(T) C T and /(B) C _B. Here AT is an arbitrary compact connected Lie 
group, /: K — > AT is a homomorphism with finite kernel, and A" = Z^(/(T)). 

Weights, roots. We identify the character group Hom(T, S* 1 ) with the weight 
lattice A* by mapping a character A to the weight dX(l)/2ni. We identify c* with 
the annihilator of in t* and [t, t]* with the annihilator of c, so that we have a 
direct sum decomposition t* — c* [f, 6]*. 

The roots are the weights of the complexified adjoint representation g. Over R 
they span the subspace t* n [t, 6]* of t*. Given a base S 1 of i?, the set of fundamental 
coweights is the basis { a* | a G 5 } of t n [6, 6] which is dual to 5. The dual root 
system R v is the root system in t fl [6, t] consisting of the dual roots or coroots a v , 
which are determined by s a X = A — A(a v )a. The set of fundamental weights is the 
basis { 7r Q | a G 5 } of t* H [6, {!]* which is dual to S v . 

If we identify t with t* via a VF-invariant inner product (•, •), then a v = 2a/ (a, a) 
and a* — 2ir a / (a, a). 



ORBITS, POLYTOPES, AND THE HILBERT-MUMFORD CRITERION 



33 



The Weyl chamber t+ is equal to c x cone{ a* \ a G S }, whereas is equal to 
c* x cone{7r Q \ a E S}. (Here cone(X) denotes the set of all nonnegative linear 
combinations of elements in X.) We define the root cone to be the cone C spanned 
by R + ; it is dual to the cone t + in the sense that A e C if and only if A(£) > for 
all £ e t+. 
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